When a full high-definition television (HDTV: High Definition Television, 1080×1920 pixels) receiver enlarges an image signal with resolution lower than that for the HDTV and displays an image thus obtained, the image becomes blurry. Similarly, when an image represented by an image signal with resolution for the HDTV is enlarged to an image with higher resolution (for example, 4K resolution of approximately 4000×2000 pixels), the image becomes blurry. As such, a conventional television receiver performs contour compensation for sharpening rise and fall of a video signal corresponding to an outline portion of an image to be displayed. In contour compensation, a high frequency component of an input image signal (a luminance signal) is extracted, amplified, and then added to the input image signal, thereby improving visual image quality.
However, conventional image enhancement processing is based on linear digital signal processing and thus incapable of generating a frequency component higher than a Nyquist frequency, i.e., a frequency component higher than ½ of a sampling frequency of a subject image. Therefore, for improvement in image quality, image sharpening by generating and using the frequency component exceeding the Nyquist frequency cannot be carried out.
The following is a description of a change in a frequency component by enlargement and enhancement processing of the image, with reference to FIG. 14. FIG. 14A illustrates a frequency spectrum of a digital image signal with a sampling frequency fs, and FIG. 14B illustrates a frequency spectrum when the digital image signal is up-converted and the number of pixels thereof is doubled in the horizontal direction. A new sampling frequency Fbs obtained through the enlargement processing doubles the original sampling frequency fs (Fbs=2·fs). Here, as illustrated in FIG. 14B, in the up-converted digital image signal, there is no frequency component between fs/2 corresponding to the Nyquist frequency of the original sampling frequency fs, and Fbs/2=fs corresponding to the new Nyquist frequency of the new sampling frequency Fbs.
FIG. 14C illustrates a frequency spectrum when, on the up-converted digital image signal, image enhancement processing employing conventional linear digital signal processing is carried out. As illustrated in the figure, due to the image enhancement processing employing the linear digital signal processing, frequency components near the original Nyquist frequency fs/2 are increased. However, the image enhancement processing employing the conventional linear digital signal processing does not generate the frequency component exceeding the original Nyquist frequency fs/2. That is, with the up-converted digital image signal, in order to improve the image quality, image sharpening by generating and using the frequency component exceeding the Nyquist frequency cannot be carried out.
As such, it has been suggested sharpening processing for generating a high-range frequency component exceeding the Nyquist frequency by nonlinear arithmetic processing (PLT 1). This sharpening processing extracts a high frequency component of an input image signal (a luminance signal) and processes the high frequency component by using a nonlinear function, thereby generating a new frequency component that does not exist in an original input image signal. This processing, as illustrated in FIG. 14D by way of example, may generate a frequency component near the new Nyquist frequency Fbs/2 exceeding the original Nyquist frequency fs/2.
However, there has been a problem that, when this sharpening processing is applied to a two-dimensional image, that is, when this sharpening processing is carried out on high frequency components in the horizontal direction and the vertical direction of the image, a phenomenon in which a diagonal line glitters occurs in an image obtained by the sharpening processing.
FIG. 15 is a diagram illustrating a configuration for consecutively performing, in the vertical direction and the horizontal direction, the sharpening processing for generating the high-range frequency component exceeding the Nyquist frequency. FIG. 16 are diagrams illustrating a frequency component of a signal at each stage. FIG. 16A illustrates a frequency component of an input image signal Sin of a digital image with a sampling frequency fh in the horizontal direction and a sampling frequency fv in the horizontal direction. The digital image has a Nyquist frequency fh/2 in the horizontal direction and a Nyquist frequency fv/2 in the vertical direction and, as illustrated in the figure, there is no frequency component in a range exceeding the Nyquist frequency. When the sharpening processing is carried out on the input image signal Sin in the vertical direction, in a signal S1 thus obtained, as illustrated in FIG. 16B, the frequency component is generated in a wide range exceeding the Nyquist frequency fv/2 in the vertical direction. When the sharpening processing is further carried out on the signal S1 in the horizontal direction, in an output image signal Sout thus obtained, as illustrated in FIG. 16C, the frequency component is generated in a wide range exceeding the Nyquist frequency fh/2 in the horizontal direction. As illustrated in the figure, regions at four corners of the frequency component of the output image signal Sout, i.e., regions at high frequencies in both the horizontal direction and the vertical direction are subjected to the sharpening processing in the horizontal direction and the vertical direction in an overlapping manner, whereby the glitter of the image is emphasized.
In order to clear such glitter, a technique for providing a two-dimensional filter at a preceding stage of horizontal direction sharpening processing and vertical direction sharpening processing has been proposed (see PLT 2).
The PLT 2,as illustrated in FIG. 17, provides the two-dimensional filter at a preceding stage of the horizontal direction sharpening processing and the vertical direction sharpening processing. FIG. 18 is a diagram illustrating an example of frequency characteristics of the two-dimensional filter. As illustrated in the figure, the two-dimensional filter is characteristic in attenuating high frequency components in the horizontal direction and the vertical direction of the input image signal Sin.
The output image signal Sout generated by a circuit in FIG. 17 is less deteriorated than a signal in FIG. 16C. However, since in a region at high frequencies in both the horizontal direction and the vertical direction a harmonic is further generated in the vertical direction to a signal in which a harmonic is generated in the horizontal direction, there is a problem that an image is still likely to become glistening/flickering. Also, in order to clear the glittering/flickering, when a passage region of the two-dimensional filter is set to be narrow, the signal components subjected to the sharpening is reduced, causing a problem that effective sharpening is prohibited.
As such, we have already suggested an image processing apparatus and an image processing method capable of sharpening an image without generating a frequency component caused by the sharpening processing in the horizontal direction and the vertical direction in an overlapping manner in a frequency domain exceeding both frequency components in the horizontal direction and the vertical direction of the input image.
That is, our preceding Japanese patent application No. 2013-035186 suggests an image processing apparatus, as illustrated in FIG. 19 and FIG. 20, for sharpening the input image by generating a frequency component higher than the frequency component contained in the input image signal representing the input image, the image processing apparatus including: a vertical filter (or a two-dimensional filter) for removing a high frequency portion of a frequency component in the vertical direction of the input image signal from at least a high frequency portion in the horizontal direction contained in the input image signal; a horizontal sharpening processing unit FEh for generating a harmonic in the horizontal direction containing a frequency component higher than a frequency component in the horizontal direction contained in the input image signal; a horizontal filter (or the two-dimensional filter) for removing a high frequency portion of a frequency component in the horizontal direction of the input image signal from at least a high frequency portion in the vertical direction contained in the input image signal; and a vertical sharpening processing unit FEv for generating a harmonic in the vertical direction containing a frequency component higher than a frequency component in the vertical direction contained in the input image signal, wherein a horizontal direction processing unit in which the vertical filter is disposed at a preceding stage of the horizontal sharpening processing unit FEh and a vertical direction processing unit in which the horizontal filter is disposed at a preceding stage of the vertical sharpening processing unit FEv are connected in series (FIG. 19) or in parallel (FIG. 20).
As illustrated in FIG. 21, further, we have suggested that an amplifier (an amplification factor β satisfies 0≦β≦1) connected to a subsequent stage of one of the horizontal direction processing unit and the vertical direction processing unit, which are connected in parallel, and also to a preceding stage of the other.
As a variation of the image processing apparatus in FIG. 21, also, as illustrated in FIG. 22, the horizontal direction processing unit in which the two-dimensional LPF (a horizontal filter) is disposed at a preceding stage of the horizontal sharpening processing unit FEh and the vertical direction processing unit in which the two-dimensional LPF (a vertical filter) is disposed at a preceding stage of the vertical sharpening processing unit FEv are connected to each other in parallel, and the amplifier (note that the amplification factor β satisfies 0≦β≦1) may be connected to a subsequent stage of one of the horizontal sharpening processing unit FEh and the vertical sharpening processing unit FEv and also to a preceding stage of the other (between the other sharpening processing unit and the two-dimensional LPF).
Note that the two-dimensional LPF (the horizontal filter) and the two-dimensional LPF (the vertical filter) has characteristics as illustrated in FIG. 23A and FIG. 23B, respectively. The two-dimensional LPF (the horizontal filter) in FIG. 23A is “a filter for removing a high frequency portion of a frequency component in the vertical direction of the input image signal from at least a high frequency component in the horizontal direction contained in the input image signal” and has characteristics similar to that of the vertical LPF. On the other hand, the two-dimensional LPF (the vertical filter) in FIG. 23A is “a filter for removing a high frequency portion of a frequency component in the horizontal direction of the input image signal from at least a high frequency component in the vertical direction contained in the input image signal” and has characteristics similar to that of the horizontal LPF.